An artificial intelligence model predicts the survival of solid tumour patients from imaging and clinical data.

BACKGROUND: The need for developing new biomarkers is increasing with the emergence of many targeted therapies. Artificial Intelligence (AI) algorithms have shown great promise in the medical imaging field to build predictive models. We developed a prognostic model for solid tumour patients using AI on multimodal data. PATIENTS AND METHODS: Our retrospective study included examinations of patients with seven different cancer types performed between 2003 and 2017 in 17 different hospitals. Radiologists annotated all metastases on baseline computed tomography (CT) and ultrasound (US) images. Imaging features were extracted using AI models and used along with the patients' and treatments' metadata. A Cox regression was fitted to predict prognosis. Performance was assessed on a left-out test set with 1000 bootstraps. RESULTS: The model was built on 436 patients and tested on 196 patients (mean age 59, IQR: 51-6, 411 men out of 616 patients). On the whole, 1147 US images were annotated with lesions delineation, and 632 thorax-abdomen-pelvis CTs (total of 301,975 slices) were fully annotated with a total of 9516 lesions. The developed model reaches an average concordance index of 0.71 (0.67-0.76, 95% CI). Using the median predicted risk as a threshold value, the model is able to significantly (log-rank test P value < 0.001) isolate high-risk patients from low-risk patients (respective median OS of 11 and 31 months) with a hazard ratio of 3.5 (2.4-5.2, 95% CI). CONCLUSION: AI was able to extract prognostic features from imaging data, and along with clinical data, allows an accurate stratification of patients' prognoses.

Breast nodule classification with two-dimensional ultrasound using Mask-RCNN ensemble aggregation.

PURPOSE: The purpose of this study was to create a deep learning algorithm to infer the benign or malignant nature of breast nodules using two-dimensional B-mode ultrasound data initially marked as BI-RADS 3 and 4. MATERIALS AND METHODS: An ensemble of mask region-based convolutional neural networks (Mask-RCNN) combining nodule segmentation and classification were trained to explicitly localize the nodule and generate a probability of the nodule to be malignant on two-dimensional B-mode ultrasound. These probabilities were aggregated at test time to produce final results. Resulting inferences were assessed using area under the curve (AUC). RESULTS: A total of 460 ultrasound images of breast nodules classified as BI-RADS 3 or 4 were included. There were 295 benign and 165 malignant breast nodules used for training and validation, and another 137 breast nodules images used for testing. As a part of the challenge, the distribution of benign and malignant breast nodules in the test database remained unknown. The obtained AUC was 0.69 (95% CI: 0.57-0.82) on the training set and 0.67 on the test set. CONCLUSION: The proposed deep learning solution helps classify benign and malignant breast nodules based solely on two-dimensional ultrasound images initially marked as BIRADS 3 and 4.

Three artificial intelligence data challenges based on CT and ultrasound.

PURPOSE: The 2020 edition of these Data Challenges was organized by the French Society of Radiology (SFR), from September 28 to September 30, 2020. The goals were to propose innovative artificial intelligence solutions for the current relevant problems in radiology and to build a large database of multimodal medical images of ultrasound and computed tomography (CT) on these subjects from several French radiology centers. MATERIALS AND METHODS: This year the attempt was to create data challenge objectives in line with the clinical routine of radiologists, with less preprocessing of data and annotation, leaving a large part of the preprocessing task to the participating teams. The objectives were proposed by the different organizations depending on their core areas of expertise. A dedicated platform was used to upload the medical image data, to automatically anonymize the uploaded data. RESULTS: Three challenges were proposed including classification of benign or malignant breast nodules on ultrasound examinations, detection and contouring of pathological neck lymph nodes from cervical CT examinations and classification of calcium score on coronary calcifications from thoracic CT examinations. A total of 2076 medical examinations were included in the database for the three challenges, in three months, by 18 different centers, of which 12% were excluded. The 39 participants were divided into six multidisciplinary teams among which the coronary calcification score challenge was solved with a concordance index > 95%, and the other two with scores of 67% (breast nodule classification) and 63% (neck lymph node calcifications).